Learning To Predict A Student's Performance On Problem Sets

Project Goal The goal of this project is to show that it is possible to create a system that can effectively predict whether a student will get a problem right or wrong, given the performance of other students on a set of problems which contains the target problem and given the students performance on the same set of problems minus the target problem. The reason I would find such a system useful is that I plan on making a web site that generates dynamic problems sets for my senior project. The idea is that the web site would use students history on problems to create customized optimal sets of problems for learning a particular topic. Predicting the probability that a student gets a problem right is key to developing such a system. Data For this project the Grade 7 math problems from the California SAT9 exam were used as the problem set. The data is composed of over 400,000 students, each of whom have taken 80 problems. For the predictive system to be effective, it must accurately estimate the probability of a students success on a problem, as opposed to predicting a binary value for success or failure. This is because, it would be useful when creating dynamic problem sets, to be able to specify how challenging a problem set should. Thus rather than just calculating the average number of times a prediction is wrong, error should measure the standard deviation of the difference between the success and the predicted probability of success. A measure of average absolute value of differences promotes a binary prediction while using sum of squared errors promotes an accurate estimate of probability. The measure of error therefore will be: error = sqrt (∑i:m∑j:n (yHat ij – y ij))/(m*n)) where yHat is the predicted probability of getting target problem right, and y ij is success on the ith problem on the jth test (1 if correct, 0 if wrong). The error is summed over n, because each of the n problems is used as a target and over m for each of the m students in the sample. For frame of reference the error generated by simply guessing the mean success of the target problem over all students in the training set is shown in the table below: (Note: The fact that the generalized error was smaller for the smallest training sets is idiosyncratic to the …